Protein arginine methyltransferases (PRMTs) are proved to play vital roles in chromatin remodeling, RNA metabolism, and signal transduction. Aberrant regulation of PRMT activity is associated with various pathological states such as cancer and cardiovascular disorders. Development and application of small molecule PRMT inhibitors will provide new avenues for therapeutic discovery. The combination of pharmacophore-based virtual screening methods with radioactive methylation assays provided six hits identified as inhibitors against the predominant arginine methyltransferase PRMT1 within micromolar potency. Two potent compounds, A9 and A36, exhibited the inhibitory effect by directly targeting substrate H4 other than PRMT1 and displayed even higher inhibition activity than the well-known PRMT inhibitors AMI-1. A9 significantly inhibits proliferation of castrate-resistant prostate cancer cells. Together, A9 may be a potential inhibitor against advanced hormone-independent cancers, and the work will provide clues for the future development of specific compounds that block the interaction of PRMTs with their targets.